Method for the displaying information in a motor vehicle

ABSTRACT

A method for the display of information in a motor vehicle, in which a segment of an image from a camera is displayed on a display unit and the image segment contains a road sign after analysis by a control and analysis unit.

FIELD OF THE INVENTION

The present invention concerns a method and apparatus for displayinginformation in a motor vehicle.

BACKGROUND INFORMATION

In present-day motor vehicles, absolutely no automated functions existto support the driver in recognizing road signs. Such a function has,however, a high comfort and safety value, for example, in warning thedriver of speed limits or prohibitions against passing and/or ininforming the driver of temporarily applicable limitations to hisrequest. As a practical matter, with the existing trafficinfrastructure, road signs can only be recognized and analyzed with theaid of image sensors since the information is only available foranalysis in optical form. Owing to the high comfort and safety relevanceof road sign recognition, work is being done on video-based road signrecognition. All of the known approaches are based on a limited numberof road signs to be recognized. Based on the number of signs to berecognized, the image recorded by the camera undergoes image segmentingto recognize and classify the relevant road signs. With the appropriateclassification, the information in the road sign is obtained, forexample, a 70 km/h speed limit. The classified road sign information isreported to the driver via an appropriate display area in which, forexample, the number combination “70” is displayed. The method ofanalyzing the information in the image obtained by the sensor for roadsigns and displaying the information to the driver may be problematicbecause the classification must be made very reliably. Otherwise itcannot be ruled out that the driver may be given incorrect information.It is believed that incorrect classifications may not only reduce theacceptance of such a system, as well as its reliabilty. If for example,a driver is supplied with an incorrect speed limit resulting in hisexceeding the maximum speed limit, the question arises as to who willassume the liability for damages resulting from this. Based on therelated art, incorrect classifications cannot be ruled out in therecognition of road signs.

SUMMARY OF THE INVENTION

In contrast, the method according to an exemplary embodiment of thepresent invention for the display of information has the advantage thata segment of the image obtained by the sensor is displayed directlywithout an analysis of the content of the road sign and a classificationbeing made. The analysis of the camera's image searches for the outlinesof the road signs and presents this image segment directly to thedriver. The driver himself performs the classification; the road sign ismerely pointed out to him and it is presented to him as a support.Should an error occur in the analysis, the driver is only presented animage segment without a road sign which he can simply ignore.

In an exemplary embodiment, information in addition to the road sign canbe presented if the analysis unit additionally receives information fromother vehicle sensors besides the camera, and both the time as well asthe distance since the last recognition of a road sign can be displayed.

The exemplary method, method can also record road signs withsupplemental information, for example, “only when wet” in the case ofspeed limits. The road sign is displayed with the additional sign, thepresentation in the display area not actually being able to resolve thelettering but pointing out the presence of the supplemental sign.

It is also believed that with additional warning of the driver based ona road sign classification, the transparency of the method is preserved.The driver can easily compare the acoustic, haptic, or optical warningto the driver based on the information processing by comparing it withthe image in the display area.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a vehicle with an image sensor.

FIG. 2 shows an exemplary road scene.

FIG. 3 shows an exemplary road scene with an exemplary display.

FIG. 4 shows a road scene with an exemplary display.

FIG. 5 shows an exemplary road scene with an exemplary display.

FIG. 6 shows a schematic representation of the sensor technology in themotor vehicle.

DETAILED DESCRIPTION

FIG. 1 shows the positioning of image sensor 2 within vehicle 1. Imagesensor 2 is attached as high as possible in a central position behindthe windshield, the viewing direction being in alignment with thedirection of travel. The sensor should still be located in the area ofthe windshield wipers. FIG. 2 shows a roadway 3, which, for example, isbordered by reflector posts 9. A vehicle 8 approaching from the oppositedirection is shown schematically on the opposite roadway. A traffic signcombination is installed on the right edge of the roadway. Road sign 4has a limiting additional sign 5. In addition, road sign 6 for a “RoughRoad” is also installed. From the view of the driver, the instrumentpanel 10 of his own vehicle can be seen with steering wheel 12. A screen11 is fitted into instrument panel 10 in the field of view of thedriver, and image segment 7 with the relevant road signs is recorded bythe image sensor and separated out by the image processing beingdisplayed on screen 11.

FIG. 3 shows a scene as described in FIG. 2, information 13 beingadditionally displayed in an area of the instrument panel. FIG. 4 showsan additional display form in which distance information 14 is shown inaddition. FIG. 5 shows the display form with time information 15. FIG. 6shows an exemplary implementation of the sensor configuration for driversupport. Image sensor 2 is connected via a cable to an electronicanalysis circuit 16. Electronic analysis circuit 16 has additionalconnections to additional sensors 19, for example, a touch key via whicha driver's intentions can be communicated to the electronic analysiscircuit, and vehicle sensors 18 which detect, for example, vehiclespeed, distance or time measurements. Electronic analysis circuit 16 isconnected to control unit 17 which, in turn, is connected to a displayarea 11. Image sensor 2 is positioned in the direction of vehicle traveland records video images of the vehicle surroundings. These images aresent to the electronic analysis circuit via the cable. Software must beused in the electronic analysis circuit which can separate the roadsigns needed for driver support from other image components. In general,not all road signs are separated out. This depends, however, on theselected type of driver support. If it is desired, for example, tosupport the driver only with regard to the particularly significantclass of road signs that limit vehicle speed, the software will beconfigured to recognize only those road signs that indicate a limitationof maximum speed. The software recognizes road signs on the basis oftheir outline structure 7, on the basis of colors or on the basis ofoptical structures located on the sign. Suitable algorithms for thesegmenting of road signs are provided in “Detektion von Verkehrszeichenaus monochronen Bildfolgen [Detection of Road Signs from MonochromeImage Sequences],” Master's thesis by Holger Janssen, Institute forTheoretical Information Technology, University of Hanover 1997. If aroad sign necessary for driver support has been segmented in a sensorimage, it can be segmented again in each subsequent image and trackedover time until it leaves the field of view of the sensor. By repeatedsegmenting and tracking over time, it is possible to achieve astabilization of the recognition of relevant road signs. Aftersegmenting, a classification must be carried out in a suitable mannersince it must be decided whether a road sign relevant for driver supportis present. In doing so, not only a road sign relevant for driversupport is shown with its informational content but rather the imagesegment containing the road sign is presented directly to the driver viaa display area 11. If the recognition and segmenting of road signs isincorrect, which cannot be completely ruled out with the complexity ofroad sign recognition, a randomly and incorrectly segmented scenesegment without road signs is presented to the driver in his displayarea. He can detect the malfunction of the system immediately and actaccordingly. The system does not mislead the driver into a false senseof security.

Various improvements to the system for support of the driver areadditionally possible. The image quality of the image recorded by theimage sensor can be improved under certain circumstances with the aid ofalgorithmic methods. Such methods include image brightening and/ordarkening, contrast intensification and edge sharpening. This methodmerely improves the image for the driver's view without, however,distorting or interpreting its information content.

An exemplary embodiment of the system allows the most recently segmentedroad sign from the large number of road signs relevant to the level ofdriver support desired at the time to be continuously presented to thedriver in display area 11. On expressways on which primarily speedlimitations, passing prohibitions and cancellations of such prohibitionsoccur, the driver could be informed of the most recent applicabledriving limitation by presentation of the most recent segmented roadsign.

An additional exemplary embodiment of the system makes it possible toretain a certain number of the most recently segmented road sign imagesin an electronic memory. In this case, an input device via a keypad 19must be provided for the driver which makes it possible for him tospecify which sign he would like to have displayed to him. A sequentialpresentation of the stored road sign images, starting with the mostrecently segmented sign, is particularly practical. By the input of asignal, for example, via suitable touch keys or an acoustic input via asuitable input sensor 19, the driver can switch to the next previouslysegmented road sign. It is thus possible for the driver to be remindedof the most recently recognized and segmented road signs as he wishes.In addition to the exclusive presentation of the segmented road signs,an additional character interpretation may be practical. In this way thedriver can be provided with symbolic information via defined characters13, see FIG. 3. In this case, by comparing the displayed characters withthe simultaneously presented image segment, the driver can immediatelydetermine if the automatic road sign interpretation is incorrect. Inthis case also, the function of the system is transparent so that theresponsibility remains exclusively with the driver.

In a further exemplary embodiment, warnings are output to the driverwhich can be obtained on the basis of information from additionalvehicle sensors. Thus, for example, a speed signal can be combined withthe output of a warning. If the driver is driving faster than ispermitted by the maximum allowable speed according to the road sign, theanalysis unit recognizes this discrepancy and generates an acousticwarning signal. In doing so, it is possible to activate a warning tone,an acoustic speech output or even an optical warning display.

In a further exemplary embodiment, lane recognition is implemented viathe image sensor and the analysis unit. With the aid of such image-basedlane tracking, it is possible for the analysis unit to detect when thevehicle pulls out into the opposite lane for the purpose of passing, forexample. In connection with a previously recognized passing prohibition,it is then possible to emit a warning signal again.

In addition to the presentation of the road sign on display area 11, itis also possible to supply the driver with distance informationindicating the distance between the present vehicle position and theposition at which the road sign was detected. Additional information 14is presented in the display area—see FIG. 4. Consequently, the driverhas at least a limited possibility to check if the displayed road signis at all still relevant to him. In this connection, the distanceinformation can be obtained directly from the distance measuringequipment present in every vehicle or also by associating vehicle speedinformation with the time passed since the road sign was detected. Thesensor signals are evaluated in analysis unit 16 which brings about apossible output in display area 11 via control unit 17.

As an alternative to a presentation of the distance information, timeinformation can be displayed to the driver simultaneously with thepresentation of the road sign. For this purpose, the time differencebetween detection of the road sign and the actual time must be recordedfor the driver via control unit 17.

For image detection, a CMOS sensor having a non-linear wallcharacteristic is advantageous for road sign detection. Road signsgenerally have painted, metallic surfaces or plastic surfaces that aresmooth and readily reflect incident light. With unfavorable lightincidence or night driving with headlight illumination, it is thuspossible for extreme brightness dynamics caused by reflection of lightor other effects to occur. Such brightness differences can be controlledwith the aid of a non-linear image sensor, making it possible to supplya simple preparation of an analyzable video image to the electronicanalysis circuit. In this connection, an image sensor with a logarithmiccharacteristic is particularly advantageous since it ensures a constantcontrast resolution over the entire brightness range.

What is claimed is:
 1. A method for displaying information in a motorvehicle associated with a camera that is coupled to an image recognitionarrangement for performing a control operation and an analysisoperation, the method comprising the steps of: obtaining an image of aroad sign using the camera; analyzing the image of the road sign usingthe image recognition arrangement in order to provide an image segmentcontaining the road sign; and displaying the image segment directly to adriver of the motor vehicle on a display unit.
 2. The method of claim 1,further comprising the steps of: analyzing information from at least oneadditional vehicle sensor using the image recognition arrangement; anddisplaying results of the analyzing step in addition to the imagesegment.
 3. The method of claim 2, further comprising the step ofdisplaying at least one of a distance and a time occurring since a priorrecognition of a prior road sign.
 4. The method of claim 1, furthercomprising the step of displaying a classification of a most recentlyrecognized road sign.
 5. The method of claim 1, wherein the imagesegment includes an outline of the road sign and a supplemental sign. 6.The method of claim 1, further comprising a performance of at least oneof the steps of: retrievably filing a most recently recognized road signin a database; and presenting the most recently recognized road sign tothe driver.
 7. The method of claim 1, further comprising the step ofproviding to the driver at least one of an acoustic warning and anoptical warning if a comparison of a setpoint value and an actual valuerelating to speeds of the motor vehicle indicates an excess speed. 8.The method of claim 1, further comprising the steps of: recognizing alane change by the motor vehicle using the image recognitionarrangement; and triggering at least one of an acoustic warning and anoptical warning if the lane change is in a prohibited passing zone. 9.The method of claim 1, wherein the camera includes at least one of anon-linear CMOS image sensor and a logarithmic CMOS image sensor.
 10. Amethod according to claim 1, further comprising: improving the imagesegment using an algorithmic method.
 11. A method according to claim 10,wherein the algorithmic method used includes image brightening.
 12. Amethod according to claim 10, wherein the algorithmic method usedincludes image darkening.
 13. A method according to claim 10, whereinthe algorithmic method used includes contrast intensification.
 14. Amethod according to claim 10, wherein the algorithmic method usedincludes edge sharpening.
 15. The method of claim 1, further comprising:providing an optical warning if a comparison of a setpoint value and anactual value relating to speeds of the motor vehicle indicates an excessspeed.
 16. An apparatus for displaying information in a motor vehicle,the apparatus comprising: a camera for obtaining an image of a roadsign; and an arrangement for image recognition coupled to the camera,wherein the arrangement for image recognition is adapted for analyzingthe image of the road sign and for providing an image segment containingthe road sign for display on a display unit to a driver of the motorvehicle.